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ABSTRACT 



The educational authorities of the Member States of 



the European Economic Community have begun to study the evaluation of 
vocational training programs. The study first will probe two 
strategic questions: (1) evaluation — to what end and by whom should 
evaluation be made?; and (2) what should be evaluated? The first 
program slated to be evaluated is training for women. The initial 
studies will open up a new field of work on evaluation in which 
individual cases will be studied within a general context. A 
preliminary finding in the study of women's training is that 
mathematical techniques have been only partially applied to 
evaluation techniques. Many questions can be posed in the area of 
training for women and used for evaluation of other areas. (A guide 
for the design of a vocational training evaluation is included in 
this report.) (KC) 
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evaluation was then requested to analyse the ansv;ers 
to the questionnaire and the discussions of the con- 
ference. This institute applied a theoretical evalu- 
ation matrix. At the end of the exercise, the matrix 
was modified to give more consideration to the diffe- 
rent components of the problem of equal opportunities. 
We will present this matrix later. 

Parallel to this survey which is a specific case 
study, the evaluation of in-company training has also 
been the c , of a separate study whose results 
will be ava^j.able in 1990. Some twenty companies par- 
ticipated in this operation. 

In addition to this, given the growing interest in the 
question of evaluation within the context of training 
linked to territorial development processes, the 
Centre has recently embarked on 24 case studies spread 
over a dozen regions of the Community. Here too, the 
results are expected in 1990. 

In both the projects CEDEFOP, like the majority of its 
partners collaborating in these studies, views the 
issue from the perspective of change. And change, for 
the Centre, means improvement, development of voca- 
tional training. Inspection or supervision is not, 
under any circumstance, the aim of the exercise. As 
Jean AUBEGNY said in two concise statements, what is 
involved is "not to give account but to take account** 
and to "create effective conditions for change" (in 
"The Pitfalls of Evaluation"). 

It would also be useful to underline right from the 
outset that CEDEFOP, like all modern evaluation spe- 
cialists, views the issue in its multidimensionality. 
The results of scientific analyses modelled on the 
experimental sciences, cannot as such cover the entire 
situation. Other approaches have to be developed. 

"Reseurcher/practitioner" , this could be the term 
given to the profile of CEDEFOP 's partners involved in 
the current studies on evaluation. This highlights 
another aspect of a dynamic evaluation concept: it is 
a dimension of the practical implementation of voca- 
tional training. 

In this issue of Flash we will present the first 
studies undertaken within the project "Equal oppor- 
tunities and vocational training" by following the 
matrix mentioned above. We will comment on some 
points. In any evaluation two sets of questions arise: 
evaluation by whom and to what end; evaluation^ how. 
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I - STRATEGIC QUESTIONS 



Evaluation, to what end? 

CEDEFOP*s questionnaire was addressed to practitio- 
ners/researchers providing training for women alone. 
In the answers to the question whether they know what 
use is made of the results of the evaluations carried 
out, the interviewed persons said that they sent the 
conclusions of their evaluations to national and 
Community bodies which had commissioned the evalu- 
ation; they said that they had no idea of what hap- 
pened to these conclusions. These persons think that 
their reports, extensively documented, are not utili- 
zed properly; they wonder what type of documents they 
should send in order to hold the attention of the 
users of these conclusions. In this connection a 
reference was even made to an "evaluation of the 
decision-makers" ! 

The hazards in the give-and-take of information are 
many. Evaluation has not yet become the dialectic 
process between those commissioning it, the providers 
of training, and the evaluators, which it should be if 
it is to lead to change. One should react against a 
type of "ghetto" operation because, in the case of 
training undertaken as "experimental pilot projects", 
precisely in order to test the hypothesis of change, 
the question "what happens to the results of the 
evaluation" acquires a great strategic significance. 
Evaluations should lead to the introduction of other 
methods or point out improvements needed in existing 
methods? they should influence the political decision 
taken in this field. Interaction between all the 
parties concerned is, in the present state of affairs 
still too weak or non-existent. 



Two remarks should be made in connection with this 
highly relevant question of the utilization of evalu- 
ation results : 

- the providers of training for women could incor- 
porate the goal of evaluation in a follow-up logic 
instead of a development logic. The reason for this 
is that, in many cases of training for women, che 
staff of the programmes have a non-permanent status, 
they are often recruited for a limited period of 
time because of sporadic funding. In other words, 
the permanent or non-permanent status of the trai • 
ning programme has an impact which. should be taken 
into consideration. Furthermore, other factors, not 
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necessarily specific to training programmes for 
women, could also induce the providers of training 
to try and convince their funding sources of the 
quality and the usefulness of their activities. 
These factors include: institutional re-structuring 
- as in the case of regionalization - with all the 
political and social stakes involved and budgetary 
restrictions. In other words, a competitive situ- 
ation could arise between the numerous bodies pro- 
viding a "non- regulated" form of training. 

- for the trainees themselves evaluation - control of 
acquired knowledge - has an immediate significance. 
It takes concrete form in the acquisition of a 
certificate. This document, wheh it exists, enables 
them to present themselves with self-assurance to 
local employers. Certainly the certificate is valu- 
able, but it does net attest more than a part of the 
acquired knowledge. If they have been appropriately 
trained, the evaluators of training can also appre- 
ciate the other qualities which training has deve- 
loped in women: creative capabilities and social 
skills, elements as important for their "employa- 
bility" as those attested in the certificate. 

There is a lack of diversified and appropriate crite- 
ria for the evaluation of training for women. In its 
study on in-^'ompany training CEDEFOP intends to 
broaden the range and the suitability of precisely 
these criteria. 

2. Vvhom to ask for an evaluation? 

Another complex question is "Whom to ask for an eva- 
luation of the training programmes". 

The International Labour Office (ILO) presents an 
interesting alternative. ILO addresses the request for 
evaluation to itself. Within its organization there is 
an "evaluation" cell whose tasks are diverse but which 
all have the aim of making full use of the programmes 
implemented and financed by ILO: prevent duplication 
and the repetition of errors, improve contents, 
strategies and methods in the future. 

In the case of national agencies such as ONEM in 
Belgium, AFPA in France, BIBB in the Federal Republic 
of Germany, INEM in Spain and others, there is one 
person or team in charge of evaluation questions. 
However, their mode of functioning, the nature of 
their links with training providers and the evaluators 
in the field still have to be clarified. 
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One particular case was observed during the survey: an 
association which provided training for women had 
entrusted the evaluation of its activities to a uni- 
versity team. It is felt that more objectivity and 
integrity are ensured if the evaluation is done by an 
outside body. In practice however, the sole fact of 
involving an outside agency does not give evaluation 
this characteristic of objectivity; it is necessary in 
each case to examine in detail both the material which 
has been evaluated and the formal and informal links 
which the evaluators have with the evaluated training 
courses and their organizers on the one hand, and with 
those commissioning the evaluation on the other. 

In the case of training for womsn (given by women in 
most cases) which is our subject here, another aspect 
of the overall problem stands out, namely, the in- 
volvement of the evaluators in the problems specific 
to this target group, their subjectivity vis-5-vis 
this issue. 

In other v/ords, the occupational profile of the eva- 
luator is a complex matter. It implies expertise in 
various fields: the segment of the labour market for 
which the training prepares the trainees, the corpo- 
rate world of compan ie's~ and firms, and the institu- 
tional and functional universe of education and 
training. The act of evaluating is a complex function 
which is undertaken on the basis of mult;.ple inter- 
actions and with a number of partners. A great deal of 
objectivity is required to make a synthesis of all 
this subjectivity, and a feeling for dialogue and the 
ability to listen are al-so required. No one can becomel 
an evaluatoi by improvising. Evaluators have to be 
trained. 



Through xts work with practitioner/researchers and 
evaluation experts, the Centre is trying to establish, 
a priori, a list of efficiency indices, on the one 
hand in tenris of promotion of women within the "Equal 
opportunities and vocational training" project and on 
the other hand, in terms of the economic repercussions 
of its activities under the heading "Regional deve- 
lopment and vocational training". It also tries to 
test the coherence of the evaluation method in rela- 
tion to the situation to which it is applied. If it 
can attain these two objectives, the Centre will be in 
a position to present some refer^ence vocational 
training programmes which are worthy of receiving 
financial support and being reproduced in part or in 
full. 
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II - PRACTICAL QUESTIONS; EVALUATE WHAT? 
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There is much to be evaluated (see following matrix). Let 
us take a look at two aspects because of their relevance 
for women: the problem categories and the objectives. 

1 . The problem categories 

Obviously, the issue of equal opportunity is at the 
origin of the schemes; it is the reason why training 
for women alone is envisaged. However, once this has 
been said, this aspect unfortunately does not hold the 
attention of the evaluators, and the "label" of 
equality covers various forms of reality. 
Let us take the example of a long-term training pro- 
gramme in the Federal Republic of Germany for which 
the reasons given were economic factors linked to the 
crisis, the probable shortage of specialized workers 
and the unemployment rate of young girls. In 1977, the 
year the programme was set up, the situation on the 
market for apprenticeship training places was catas- 
trophic because of economic stagnation and the arrival 
on the market for demographic reasons of a large wave 
of young people. Girls were particularly affected. 
Male occupations were opened to young girls for the 
following reasons: 

* there were (still) vacant apprenticeship training 
places in these occupations? 

* since the 1970s the feminist movement had become 
stronger and demanded, among other things, equal 
occupational opportunities for women; 

* a shortage of specialized workers was forecast; it 
was thus necessary to make women interested in 
occupations with a "high tech" content. 

France represents another case in point: it is true 
that the stimulus first came from the State which 
resolutely accepted and applied the Community direc- 
tives on equal opportunity, but the companies deve- 
loped training courses for their employees and their 
workers mainly to cope with changes and restructuring. 

However, these initial situations have not been ana- 
lysed in depth. The original objectives evolved on 
their own in the course of the development of training 
which could extend over several years. Furthermore, 
the situation on the labour market also changed during 
this period. 
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Thus, a multitude of difficulties for a posteriori 
analysis arose because of the lack of a priori 
clarity, plus the difficulty of keeping up vith a 
complex and continuously evolving situation. This is 
not only found in initiatives reserved for women. 

2. The objectives 

The national reports which, as we recall, are written 
by training providers are full of descriptions of the 
objectives assigned to training. There we find that 
the "hard core" of evaluation as it is practised con- 
sists of comparing the results obtained at the end of 
the course to the objectives fixed at the beginning 
mostly in terms of employment and professional pro- 
gress. It is the teams of training providers who can 
best identify the needs of the target group: 
professional "de-compartmentalization" , technical 
proficiency, financial know-how, a professional 
ident \ty . 

At the level of and within the teaching teams, growing 
attention is being paid to the situation of women: the 
need to identify and remove barriers to their trai- 
ning, to take into consideration their dual obliga- 
tions in the household and at work, to approach future 
employers on their Uehalf, to adopt methods which can- 
generate motivation, commitment, assumption of res- 
ponsibility, to modify along non-sexist lines the 
instruments for the recruitment of trainees and 
recognition of their acquired skills, to install a 
mode of training adapted to the socio-economic 
environment. 




The objectives may be grouped into three categories: 
those related to the exercise of an occupation, those 
related to personal development and those related to 
the institutional forms of training. 

In the first case, in addition to the acquisition of 
precise know-how, the providers of training express 
the success of their activities in terms of: learning 
to learn, to be enterprising, to resolve the problems 
related to the employment market and the working of 
the companies . 

In addition to this bedrock of knowledge, training is 
also expected to bring out qualities which are consi- 
dered to be a part of personal well-being: self-con- 
fidence, a positive image of oneself; ability to com- 
municate, negotiate, make a better analysis of one's 
own problems, work in synergy with one's environment, 
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Lastly, training programmes are judged by the certi- 
ficates which they award and by the access they enable 
to continuing training. Here, it is a question of re- 
launching a career, a professional future which was 
blocked up to then, or of encouraging mobility. 



-Conclusion- 

By requesting CEDEFOP to embark on the task of evaluation 
of training for women, the EC Commission induced it to 
apply its know-how in a highly specifir: situation. 

In the 1980s training schemes for women were justified by 
the threat or the fact of unemployment which affected 
women more than men. Because they lacked technical skills 
women remained on che margin of technological progress. 
And, in any case, the companies did not base their deve- 
lopment on their training. As the author of a recent study 
on t'le "Women-Technologies'* binomial requested by the 
Commission wrote, women are the "feeble part" of the 
companies. Under these circumstances, the evaluation of 
training for women would not attain its full sense unless 
it were re-located within the context of the evaluation of 
training activities as a whole. 

During the 1980s training for women inside and outside the 
companies was too inadequate in quantity and quality to 
respond to demand in an even remotely coherent manner. The 
field of evaluation is narrow. It is necessary to compare 
the obtained results with other results and to place the 
conclusions drawn from evaluations of pilot schemes within 
the overall context of evaluation of the normal courses 
offered in vocational training. 

Thus, as far as CEDEFOP is concerned, its initial studies 
will lead it to open up a new field of work on evaluation 
where individual cases will be studied within a general 
context . 

In order to do this, without relapsing into the illusion 
which leads to the belief that the whole answer to the 
questions of evaluation lies in a scientific approach 
modelled on the exact sciences, it is necessary to have 
comparable, sex-differentiated data in the field of voca- 
tional training. Through its work on the financing of 
vocational training, the Centre is examining these basic 
elements. The statistical approach and the qualitative 
case-study approach are complementary to one another. 
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A striking fact which emerged is that the persons inter- 
viewed by CEDEFOP within the context of its work on women 
only partially apply mathematical evaluation techniques to 
define the cost-effectiveness ratio. Are they placing 
themselves in a perspective of change rather than control? 
Without falling into the trap of thinking that vocational 
training is always a better solution than unemployment and 
that the latter per se justifies training at any cost 
whatsoever, it should be underlined that the analysis of 
the cost-effectiveness ratio is usually a part of the 
control logic: it is necessary to render account. If one 
can prove through intricate formulas that three months of 
training at a specific level of qualification costs less 
than the amount paid during the same period to a person at 
the same professional level, then this is certainly an 
indispensable argument, but it should be placed within a 
larger configuration of questions. 

Also, for women; 

- training, in a company, mostly represents a form of 
selection; what are the e* Jects of the selective 
approach on the "collective group" of women in the 
company? 

- training implies occupational integration; to what 
extent have new jobs been envisaged and are the male 
colleagues prepared to work with women on an equal 
footing? 

- training provides a better basis for inter-company and 
intra-company mobility; what validation is envisaged so 
that women can find a job outside the company which has 
trained them? 

- apart from the necessity of earning a living, is there a 
"future" for women in the post-1992 world of work? Will 
they be full partners within the company? 

- can training be something other than an assimilation 
force: i.e. the means to "enter" the labour market and 
function well there? Will women have no other choice, as 
in the past, but to adapt themselves and conform to its 
laws and demands? 

Can the shortage of manpower forecast by some not be the 
opportunity to create new areas of negotiation: training^ 
yes. employment, yes, but not at the cost of the multiple 
possibilities of intervention in their content, their 
significance, their organization. The advances in equal 
opportunity awaited by many women should lead to a 
different society. 
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GUIDE FOR THE DESIGN 
OF AM EVALUATION OF VOCATIONAL TRAINING 

Phase I: Answer the strategic questions of evaluation 

WHO ASKS WHOM FOR 'AN EVALUATION? 

The funding body? The providing body? The 
evaluator? 

TO DO WHAT? 

To define the logic (s) of the USE of evaluation 

Economic logic; evaluation should serve to 

- control the use of grants? 

- obtain additional grants? 

If yes, what are the criteria applied by the 
funding authorities to apply control? 
what documents have to be supplied? 

Logic of SOCIAL CHANGE : should evaluation serve 
to 

- obtain useful information to define training 
policies? 

- participate in the definition of training 
policies? 

Logic of INTERNAL FUNCTIONING ; should evaluation 
be a process of aid for the decisions 
accompanying the project? 
if yes, what questions should it answer? 

what presence in the field is 

contemplated? 

what accompanying approach is 
contemplated? 

Logic of PEDAGOGICS : 

Is an evaluation of the skills acquired by the 
trainees envisaged? 

if yes, who is responsible for this evaluation? 

are the trainers trained for evaluation? 
do the trainees participate in this 
evaluation 

and in the construction of evaluation 
tools? 



Phase 2: Establish the budget allocated to evaluation 
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Phase 3: Answer practical questions 

Against the background of the selected logics and 
the priorities accorded to each one of them, and 
the grants earmarked for evaluation, answer the 
following questions: 

EVALUATE WHAT? 

1 . The problem categories of the programme 
What economic .reasons were given to justify 
the training? 

What reasons related to the emancipation of 
women were given to justify the training? 

2. The objectives of the programme 
What are the objectives of training? 

- for women, in terms of 

* ability to exercise an occupation 

* personal development 

* the training path 

- for the training agency, in terms of 

* economics 

* the development of practical methods and 
increase of knowledge 

- for the companies, in terms of 

* economics 

* development and improved functioning of 
the firm 

- for the partner organizations in training, 
in terms of 

* economics 

* development 

- for society as a whole, in terms of 

* social change 

How do the objectives evolve? 
3* Training activities 

What type of training 

- leading to qualifications? 

- for personal development? 
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What methods of work 

- individualization 

- sma) ' group? 

- lar group? 

- in-corapany training periods? 
What teaching tools? 

What teaching staff? 

What trainee participation? 

What evaluation of acquired skills? 

What structures for placement, job-seeking, 
guidance, social and family aid,... 

4. The organization of the programmes 
Geographical location? 
Organization in time? 

Venues of trainir '> 

Management of financial r .ources? 

Management of human resources? 

Management of information and communication? 

Arrangemepts for child care? 

5. Relations with the environment 
What policy of access to training? 
What collaboration with other agencies? 
What accompanying committee? 

What relations with the companies? 

What political/ social, economic context? 

6. The teaching staff 
Number of teachers? 
Skills of the trainers? 
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What ratio of men to women? 

What type of teaching teams (tutors, 
steering group, mult^-disciplinary team?) 

What climate '>f work? 

What status, what employment contracts? 
What prospects? 

7 . The target groups for training 
Age-groups? 

Situation in relation to employment? 
Family situation? 

Skills or qualifications at the start? 
Attitudes to' training? 

Attitudes of the social environment (family, 
group) to training? 

8. Evaluation 

What request to vhom? 

What evaluation logic? Why? 

What participation of whom in evaluation? 

What fields of research selected? 

What evaluation strategies? 

Who uses the results of evaluation? 

What satisfaction with respect to 
evaluation? 

9. The results of the programme 

What are the results obtained (quantitative 
and qualitative) 

- for women, in terms ^f 

* ability to exercise an occupation 

* personal development 

* the training path 
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- for the training agency, in terms of 

* economics 

* the development of methods 

* the increase of knowledge 

- for the companies, in terms of 

* economics 

* improved functioning of the firm 

- for the partner organizations in training, 
in terms of 

* economics 

* development of methods, acquisition of 
knowledge. 

What are the relations between the obtained 
results, the envisaged objectives and the 
expressed needs? 

WHEN TO EVALUATE? 

1. Before training starts? 

2. During training? 

3. When training ends? 

4. After training? 

HOW TO EVALUATE? 

1. Contemplate what? 

the logic of evaluation 

- the questions which evaluation must answer 

- the evaluation strategy 

- the presence of the evaluator 

- the documents produced for evaluation 

- the dissemination of the evaluation 
results 

2. Collect data (which methods to be used?) 

3. Analyse the data (what validity control to 
be applied?) 

4. Communicate the results (to whom and how?). 
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